{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {},
  "cells": [
    {
      "metadata": {},
      "source": [
        "<td>\n",
        "   <a target=\"_blank\" href=\"https://labelbox.com\" ><img src=\"https://labelbox.com/blog/content/images/2021/02/logo-v4.svg\" width=256/></a>\n",
        "</td>"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "<td>\n",
        "<a href=\"https://colab.research.google.com/github/Labelbox/labelbox-python/blob/master/examples/annotation_import/image.ipynb\" target=\"_blank\"><img\n",
        "src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
        "</td>\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "<td>\n",
        "<a href=\"https://github.com/Labelbox/labelbox-python/tree/master/examples/annotation_import/image.ipynb\" target=\"_blank\"><img\n",
        "src=\"https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white\" alt=\"GitHub\"></a>\n",
        "</td>"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Image annotation import\n",
        "This notebook will provide examples of each supported annotation type for image assets.\n",
        "\n",
        "### [Model-assisted labeling (MAL)](https://docs.labelbox.com/docs/model-assisted-labeling)\n",
        "\n",
        "* This workflow allows you to import computer-generated predictions (or simply annotations created outside of Labelbox) as pre-labels on an asset.\n",
        "\n",
        "The imported annotations will be pre-populated in the labeling editor. However, in order to convert the pre-labels to real annotations, a human labeler will still need to open the Data Row in the Editor and submit it. This functionality is designed to speed up human labeling.\n",
        "\n",
        "### [Import ground truth](https://docs.labelbox.com/docs/import-ground-truth)\n",
        "\n",
        "* This  workflow functionality allows you to bulk import your ground truth annotations from an external or third-party labeling system into Labelbox Annotate. Using the label import API to import external data is a useful way to consolidate and migrate all annotations into Labelbox as a single source of truth.\n",
        "\n",
        "### Python annotation types vs NDJSON\n",
        "**Python annotation type (recommended)**\n",
        "- Provides a seamless transition between third-party platforms, machine learning pipelines, and Labelbox.\n",
        "\n",
        "- Allows you to build annotations locally with local file paths, numpy arrays, or URLs\n",
        "\n",
        "- Easily convert Python Annotation Type format to NDJSON format to quickly import annotations to Labelbox\n",
        "\n",
        "- It supports one-level nested classification (free text / radio / checklist) under the object or classification annotation.\n",
        "\n",
        "**NDJSON**\n",
        "- Skip formatting annotation payload in the Python Annotation Types format just to convert back to NDJSON\n",
        "\n",
        "- Ability to create the payload in the NDJSON import format directly\n",
        "\n",
        "- It supports any levels of nested classification (free text / radio / checklist) under the object or classification annotation."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "## Setup"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "!pip install -q \"labelbox[data]\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "import uuid\n",
        "from PIL import Image\n",
        "import requests\n",
        "import base64\n",
        "import labelbox as lb\n",
        "import labelbox.types as lb_types\n",
        "from io import BytesIO\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Replace with your API key\n",
        "\n",
        "Guides on [Create an API key](https://docs.labelbox.com/docs/create-an-api-key)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "API_KEY =\"\"\n",
        "client = lb.Client(API_KEY)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Supported annotations for image\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "### Classification : Radio (single-choice)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "radio_annotation = lb_types.ClassificationAnnotation(\n",
        "    name=\"radio_question\",\n",
        "    value=lb_types.Radio(answer=lb_types.ClassificationAnswer(\n",
        "        name=\"second_radio_answer\")))\n",
        "\n",
        "# NDJSON\n",
        "radio_annotation_ndjson = {\n",
        "    \"name\": \"radio_question\",\n",
        "    \"answer\": {\n",
        "        \"name\": \"second_radio_answer\"\n",
        "    }\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Classification: Checklist (multi-choice)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "checklist_annotation = lb_types.ClassificationAnnotation(\n",
        "    name=\"checklist_question\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Checklist(answer=[\n",
        "        lb_types.ClassificationAnswer(name=\"first_checklist_answer\"),\n",
        "        lb_types.ClassificationAnswer(name=\"second_checklist_answer\")\n",
        "    ]))\n",
        "\n",
        "# NDJSON\n",
        "checklist_annotation_ndjson = {\n",
        "    \"name\": \"checklist_question\",\n",
        "    \"answer\": [{\n",
        "        \"name\": \"first_checklist_answer\"\n",
        "    }, {\n",
        "        \"name\": \"second_checklist_answer\"\n",
        "    }]\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Classification: Nested radio and checklist\n",
        "\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "nested_radio_annotation = lb_types.ClassificationAnnotation(\n",
        "  name=\"nested_radio_question\",\n",
        "  value=lb_types.Radio(\n",
        "    answer=lb_types.ClassificationAnswer(\n",
        "      name=\"first_radio_answer\",\n",
        "      classifications=[\n",
        "        lb_types.ClassificationAnnotation(\n",
        "          name=\"sub_radio_question\",\n",
        "          value=lb_types.Radio(\n",
        "            answer=lb_types.ClassificationAnswer(\n",
        "              name=\"first_sub_radio_answer\"\n",
        "            )\n",
        "          )\n",
        "        )\n",
        "      ]\n",
        "    )\n",
        "  )\n",
        ")\n",
        "# NDJSON\n",
        "nested_radio_annotation_ndjson = {\n",
        "    \"name\": \"nested_radio_question\",\n",
        "    \"answer\": {\n",
        "        \"name\": \"first_radio_answer\",\n",
        "          \"classifications\": [{\n",
        "              \"name\": \"sub_radio_question\",\n",
        "              \"answer\": {\n",
        "                  \"name\": \"first_sub_radio_answer\"\n",
        "              }\n",
        "          }]\n",
        "    }\n",
        "}\n",
        "\n",
        "\n",
        "nested_checklist_annotation = lb_types.ClassificationAnnotation(\n",
        "  name=\"nested_checklist_question\",\n",
        "  value=lb_types.Checklist(\n",
        "    answer=[lb_types.ClassificationAnswer(\n",
        "      name=\"first_checklist_answer\",\n",
        "      classifications=[\n",
        "        lb_types.ClassificationAnnotation(\n",
        "          name=\"sub_checklist_question\",\n",
        "          value=lb_types.Checklist(\n",
        "            answer=[lb_types.ClassificationAnswer(\n",
        "            name=\"first_sub_checklist_answer\"\n",
        "          )]\n",
        "        ))\n",
        "      ]\n",
        "    )]\n",
        "  )\n",
        ")\n",
        "\n",
        "nested_checklist_annotation_ndjson = {\n",
        "    \"name\": \"nested_checklist_question\",\n",
        "    \"answer\": [{\n",
        "        \"name\": \"first_checklist_answer\",\n",
        "        \"classifications\": [{\n",
        "            \"name\": \"sub_checklist_question\",\n",
        "            \"answer\": {\n",
        "                \"name\": \"first_sub_checklist_answer\"\n",
        "            }\n",
        "        }]\n",
        "    }]\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Classification: Free-form text"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "text_annotation = lb_types.ClassificationAnnotation(\n",
        "    name=\"free_text\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Text(answer=\"sample text\"))\n",
        "\n",
        "# NDJSON\n",
        "text_annotation_ndjson = {\n",
        "    \"name\": \"free_text\",\n",
        "    \"answer\": \"sample text\",\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Relationship with bounding box\n",
        "> **NOTE:**  \n",
        "> Only supported for MAL imports"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python Annotation\n",
        "bbox_source = lb_types.ObjectAnnotation(\n",
        "    name=\"bounding_box\",\n",
        "    value=lb_types.Rectangle(\n",
        "        start=lb_types.Point(x=2096, y=1264),\n",
        "        end=lb_types.Point(x=2240, y=1689),\n",
        "    ),\n",
        ")\n",
        "\n",
        "bbox_target = lb_types.ObjectAnnotation(\n",
        "    name=\"bounding_box\",\n",
        "    value=lb_types.Rectangle(\n",
        "        start=lb_types.Point(x=2272, y=1346),\n",
        "        end=lb_types.Point(x=2416, y=1704),\n",
        "    ),\n",
        ")\n",
        "\n",
        "relationship = lb_types.RelationshipAnnotation(\n",
        "    name=\"relationship\",\n",
        "    value=lb_types.Relationship(\n",
        "        source=bbox_source,\n",
        "        target=bbox_target,\n",
        "        type=lb_types.Relationship.Type.UNIDIRECTIONAL,\n",
        "    ))\n",
        "\n",
        "## Only supported for MAL imports\n",
        "uuid_source = str(uuid.uuid4())\n",
        "uuid_target = str(uuid.uuid4())\n",
        "\n",
        "bbox_source_ndjson = {\n",
        "    \"uuid\": uuid_source,\n",
        "    \"name\": \"bounding_box\",\n",
        "    \"bbox\": {\n",
        "        \"top\": 1264.0,\n",
        "        \"left\": 2096.0,\n",
        "        \"height\": 425.0,\n",
        "        \"width\": 144.0\n",
        "    }\n",
        "}\n",
        "\n",
        "bbox_target_ndjson = {\n",
        "    \"uuid\": uuid_target,\n",
        "    \"name\": \"bounding_box\",\n",
        "    \"bbox\": {\n",
        "        \"top\": 1346.0,\n",
        "        \"left\": 2272.0,\n",
        "        \"height\": 358.0,\n",
        "        \"width\": 144.0\n",
        "    }\n",
        "}\n",
        "\n",
        "relationship_ndjson = {\n",
        "    \"name\": \"relationship\",\n",
        "    \"relationship\": {\n",
        "        \"source\": uuid_source,\n",
        "        \"target\": uuid_target,\n",
        "        \"type\": \"unidirectional\"\n",
        "    }\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Bounding box"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "bbox_annotation = lb_types.ObjectAnnotation(\n",
        "    name=\"bounding_box\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Rectangle(\n",
        "        start=lb_types.Point(x=1690, y=977),  #  x = left, y = top\n",
        "        end=lb_types.Point(x=1915, y=1307),  # x= left + width , y = top + height\n",
        "    ))\n",
        "\n",
        "# NDJSON\n",
        "bbox_annotation_ndjson = {\n",
        "    \"name\": \"bounding_box\",\n",
        "    \"bbox\": {\n",
        "        \"top\": 977,\n",
        "        \"left\": 1690,\n",
        "        \"height\": 330,\n",
        "        \"width\": 225\n",
        "    }\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Bounding box with nested classification"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "bbox_with_radio_subclass_annotation = lb_types.ObjectAnnotation(\n",
        "    name=\"bbox_with_radio_subclass\",\n",
        "    value=lb_types.Rectangle(\n",
        "        start=lb_types.Point(x=541, y=933),  #  x = left, y = top\n",
        "        end=lb_types.Point(x=871, y=1124),  # x= left + width , y = top + height\n",
        "    ),\n",
        "    classifications=[\n",
        "        lb_types.ClassificationAnnotation(\n",
        "            name=\"sub_radio_question\",\n",
        "            value=lb_types.Radio(answer=lb_types.ClassificationAnswer(\n",
        "                name=\"first_sub_radio_answer\")))\n",
        "    ])\n",
        "\n",
        "## NDJSON\n",
        "bbox_with_radio_subclass_ndjson = {\n",
        "    \"name\": \"bbox_with_radio_subclass\",\n",
        "    \"classifications\": [{\n",
        "        \"name\": \"sub_radio_question\",\n",
        "        \"answer\": {\n",
        "            \"name\": \"first_sub_radio_answer\"\n",
        "        }\n",
        "    }],\n",
        "    \"bbox\": {\n",
        "        \"top\": 933,\n",
        "        \"left\": 541,\n",
        "        \"height\": 191,\n",
        "        \"width\": 330\n",
        "    }\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Polygon"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "polygon_annotation = lb_types.ObjectAnnotation(\n",
        "    name=\"polygon\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Polygon(  # Coordinates for the vertices of your polygon\n",
        "        points=[\n",
        "            lb_types.Point(x=1489.581, y=183.934),\n",
        "            lb_types.Point(x=2278.306, y=256.885),\n",
        "            lb_types.Point(x=2428.197, y=200.437),\n",
        "            lb_types.Point(x=2560.0, y=335.419),\n",
        "            lb_types.Point(x=2557.386, y=503.165),\n",
        "            lb_types.Point(x=2320.596, y=503.103),\n",
        "            lb_types.Point(x=2156.083, y=628.943),\n",
        "            lb_types.Point(x=2161.111, y=785.519),\n",
        "            lb_types.Point(x=2002.115, y=894.647),\n",
        "            lb_types.Point(x=1838.456, y=877.874),\n",
        "            lb_types.Point(x=1436.53, y=874.636),\n",
        "            lb_types.Point(x=1411.403, y=758.579),\n",
        "            lb_types.Point(x=1353.853, y=751.74),\n",
        "            lb_types.Point(x=1345.264, y=453.461),\n",
        "            lb_types.Point(x=1426.011, y=421.129)\n",
        "        ]))\n",
        "\n",
        "# NDJSON\n",
        "polygon_annotation_ndjson = {\n",
        "  \"name\": \"polygon\",\n",
        "  \"polygon\": [\n",
        "    {\"x\": 1489.581, \"y\": 183.934},\n",
        "    {\"x\": 2278.306, \"y\": 256.885},\n",
        "    {\"x\": 2428.197, \"y\": 200.437},\n",
        "    {\"x\": 2560.0, \"y\": 335.419},\n",
        "    {\"x\": 2557.386, \"y\": 503.165},\n",
        "    {\"x\": 2320.596, \"y\": 503.103},\n",
        "    {\"x\": 2156.083, \"y\": 628.943},\n",
        "    {\"x\": 2161.111, \"y\": 785.519},\n",
        "    {\"x\": 2002.115, \"y\": 894.647},\n",
        "    {\"x\": 1838.456, \"y\": 877.874},\n",
        "    {\"x\": 1436.53, \"y\": 874.636},\n",
        "    {\"x\": 1411.403, \"y\": 758.579},\n",
        "    {\"x\": 1353.853, \"y\": 751.74},\n",
        "    {\"x\": 1345.264, \"y\": 453.461},\n",
        "    {\"x\": 1426.011, \"y\": 421.129},\n",
        "    {\"x\": 1489.581, \"y\": 183.934}\n",
        "  ]\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Composite mask upload using different mask tools from the project's ontology\n",
        "This example shows how to assigned different annotations (mask instances) from a composite mask using different mask tools"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# First we need to extract all the unique colors from the composite mask\n",
        "def extract_rgb_colors_from_url(image_url):\n",
        "    response = requests.get(image_url)\n",
        "    img = Image.open(BytesIO(response.content))\n",
        "\n",
        "    colors = set()\n",
        "    for x in range(img.width):\n",
        "        for y in range(img.height):\n",
        "            pixel = img.getpixel((x, y))\n",
        "            if pixel[:3] != (0,0,0):\n",
        "                colors.add(pixel[:3])  # Get only the RGB values\n",
        "\n",
        "    return colors"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "\n",
        "cp_mask_url = \"https://storage.googleapis.com/labelbox-datasets/image_sample_data/composite_mask.png\"\n",
        "colors = extract_rgb_colors_from_url(cp_mask_url)\n",
        "response = requests.get(cp_mask_url)\n",
        "\n",
        "mask_data = lb.types.MaskData(im_bytes=response.content) # You can also use \"url\" instead of img_bytes to pass the PNG mask url.\n",
        "rgb_colors_for_mask_with_text_subclass_tool = [(73, 39, 85), (111, 87, 176), (23, 169, 254)]\n",
        "\n",
        "cp_mask = []\n",
        "for color in colors:\n",
        "  # We are assigning the color related to the mask_with_text_subclass tool by identifying the unique RGB colors\n",
        "  if color in rgb_colors_for_mask_with_text_subclass_tool:\n",
        "    cp_mask.append(\n",
        "      lb_types.ObjectAnnotation(\n",
        "        name = \"mask_with_text_subclass\", # must match your ontology feature\"s name\n",
        "        value=lb_types.Mask(\n",
        "          mask=mask_data,\n",
        "          color=color),\n",
        "        classifications=[\n",
        "          lb_types.ClassificationAnnotation(\n",
        "            name=\"sub_free_text\",\n",
        "            value=lb_types.Text(answer=\"free text answer sample\")\n",
        "          )]\n",
        "      )\n",
        "  )\n",
        "  else:\n",
        "     # Create ObjectAnnotation for other masks\n",
        "    cp_mask.append(\n",
        "      lb_types.ObjectAnnotation(\n",
        "        name=\"mask\",\n",
        "        value=lb_types.Mask(\n",
        "            mask=mask_data,\n",
        "            color=color\n",
        "        )\n",
        "    )\n",
        "  )\n",
        "\n",
        "\n",
        "# NDJSON using bytes array\n",
        "cp_mask_ndjson = []\n",
        "\n",
        "#Using bytes array.\n",
        "response = requests.get(cp_mask_url)\n",
        "im_bytes = base64.b64encode(response.content).decode('utf-8')\n",
        "for color in colors:\n",
        "  if color in rgb_colors_for_mask_with_text_subclass_tool:\n",
        "    cp_mask_ndjson.append({\n",
        "      \"name\": \"mask_with_text_subclass\",\n",
        "      \"mask\": {\"imBytes\": im_bytes,\n",
        "      \"colorRGB\": color },\n",
        "      \"classifications\":[{\n",
        "        \"name\": \"sub_free_text\",\n",
        "        \"answer\": \"free text answer\"\n",
        "      }]\n",
        "    }\n",
        "  )\n",
        "  else:\n",
        "    cp_mask_ndjson.append({\n",
        "          \"name\": \"mask\",\n",
        "          \"classifications\": [],\n",
        "          \"mask\": {\n",
        "            \"imBytes\": im_bytes,\n",
        "            \"colorRGB\": color\n",
        "      }\n",
        "    }\n",
        "  )\n",
        "\n",
        "\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Point"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "point_annotation = lb_types.ObjectAnnotation(\n",
        "    name=\"point\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Point(x=1166.606, y=1441.768),\n",
        ")\n",
        "\n",
        "# NDJSON\n",
        "point_annotation_ndjson = {\n",
        "    \"name\": \"point\",\n",
        "    \"classifications\": [],\n",
        "    \"point\": {\n",
        "        \"x\": 1166.606,\n",
        "        \"y\": 1441.768\n",
        "    }\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### Polyline"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Python annotation\n",
        "polyline_annotation = lb_types.ObjectAnnotation(\n",
        "    name=\"polyline\",  # must match your ontology feature\"s name\n",
        "    value=lb_types.Line(  # Coordinates for the keypoints in your polyline\n",
        "        points=[\n",
        "            lb_types.Point(x=2534.353, y=249.471),\n",
        "            lb_types.Point(x=2429.492, y=182.092),\n",
        "            lb_types.Point(x=2294.322, y=221.962),\n",
        "            lb_types.Point(x=2224.491, y=180.463),\n",
        "            lb_types.Point(x=2136.123, y=204.716),\n",
        "            lb_types.Point(x=1712.247, y=173.949),\n",
        "            lb_types.Point(x=1703.838, y=84.438),\n",
        "            lb_types.Point(x=1579.772, y=82.61),\n",
        "            lb_types.Point(x=1583.442, y=167.552),\n",
        "            lb_types.Point(x=1478.869, y=164.903),\n",
        "            lb_types.Point(x=1418.941, y=318.149),\n",
        "            lb_types.Point(x=1243.128, y=400.815),\n",
        "            lb_types.Point(x=1022.067, y=319.007),\n",
        "            lb_types.Point(x=892.367, y=379.216),\n",
        "            lb_types.Point(x=670.273, y=364.408),\n",
        "            lb_types.Point(x=613.114, y=288.16),\n",
        "            lb_types.Point(x=377.559, y=238.251),\n",
        "            lb_types.Point(x=368.087, y=185.064),\n",
        "            lb_types.Point(x=246.557, y=167.286),\n",
        "            lb_types.Point(x=236.648, y=285.61),\n",
        "            lb_types.Point(x=90.929, y=326.412)\n",
        "        ]),\n",
        ")\n",
        "\n",
        "# NDJSON\n",
        "polyline_annotation_ndjson = {\n",
        "  \"name\": \"polyline\",\n",
        "  \"classifications\": [],\n",
        "  \"line\": [\n",
        "    {\"x\": 2534.353, \"y\": 249.471},\n",
        "    {\"x\": 2429.492, \"y\": 182.092},\n",
        "    {\"x\": 2294.322, \"y\": 221.962},\n",
        "    {\"x\": 2224.491, \"y\": 180.463},\n",
        "    {\"x\": 2136.123, \"y\": 204.716},\n",
        "    {\"x\": 1712.247, \"y\": 173.949},\n",
        "    {\"x\": 1703.838, \"y\": 84.438},\n",
        "    {\"x\": 1579.772, \"y\": 82.61},\n",
        "    {\"x\": 1583.442, \"y\": 167.552},\n",
        "    {\"x\": 1478.869, \"y\": 164.903},\n",
        "    {\"x\": 1418.941, \"y\": 318.149},\n",
        "    {\"x\": 1243.128, \"y\": 400.815},\n",
        "    {\"x\": 1022.067, \"y\": 319.007},\n",
        "    {\"x\": 892.367, \"y\": 379.216},\n",
        "    {\"x\": 670.273, \"y\": 364.408},\n",
        "    {\"x\": 613.114, \"y\": 288.16},\n",
        "    {\"x\": 377.559, \"y\": 238.251},\n",
        "    {\"x\": 368.087, \"y\": 185.064},\n",
        "    {\"x\": 246.557, \"y\": 167.286},\n",
        "    {\"x\": 236.648, \"y\": 285.61},\n",
        "    {\"x\": 90.929, \"y\": 326.412}\n",
        "  ]\n",
        "}"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# End-to-end example: Import pre-labels or ground truth"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "## Step 1: Import data rows into catalog\n",
        "\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# send a sample image as batch to the project\n",
        "global_key = \"2560px-Kitano_Street_Kobe01s5s4110.jpeg\"\n",
        "\n",
        "test_img_url = {\n",
        "    \"row_data\":\n",
        "        \"https://storage.googleapis.com/labelbox-datasets/image_sample_data/2560px-Kitano_Street_Kobe01s5s4110.jpeg\",\n",
        "    \"global_key\":\n",
        "        global_key\n",
        "}\n",
        "\n",
        "dataset = client.create_dataset(name=\"image-demo-dataset\")\n",
        "task = dataset.create_data_rows([test_img_url])\n",
        "task.wait_till_done()\n",
        "\n",
        "print(f\"Failed data rows: {task.failed_data_rows}\")\n",
        "print(f\"Errors: {task.errors}\")\n",
        "\n",
        "if task.errors:\n",
        "    for error in task.errors:\n",
        "        if 'Duplicate global key' in error['message'] and dataset.row_count == 0:\n",
        "            # If the global key already  exists in the workspace the dataset will be created empty, so we can delete it.\n",
        "            print(f\"Deleting empty dataset: {dataset}\")\n",
        "            dataset.delete()\n",
        "\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "print(dataset)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Step 2: Create/select an ontology\n",
        "Your project should have the correct ontology setup with all the tools and classifications supported for your annotations and the tool names and classification instructions should match the `name` fields in your annotations to ensure the correct feature schemas are matched.\n",
        "\n",
        "For example, when we created the bounding box annotation above, we provided the `name` as `bounding_box`. Now, when we setup our ontology, we must ensure that the name of the bounding box tool is also `bounding_box`. The same alignment must hold true for the other tools and classifications we create in our ontology."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "ontology_builder = lb.OntologyBuilder(\n",
        "    classifications=[  # List of Classification objects\n",
        "        lb.Classification(class_type=lb.Classification.Type.RADIO,\n",
        "                          name=\"radio_question\",\n",
        "                          options=[\n",
        "                              lb.Option(value=\"first_radio_answer\"),\n",
        "                              lb.Option(value=\"second_radio_answer\")\n",
        "                          ]),\n",
        "        lb.Classification(class_type=lb.Classification.Type.CHECKLIST,\n",
        "                          name=\"checklist_question\",\n",
        "                          options=[\n",
        "                              lb.Option(value=\"first_checklist_answer\"),\n",
        "                              lb.Option(value=\"second_checklist_answer\")\n",
        "                          ]),\n",
        "        lb.Classification(class_type=lb.Classification.Type.TEXT,\n",
        "                          name=\"free_text\"),\n",
        "        lb.Classification(\n",
        "            class_type=lb.Classification.Type.RADIO,\n",
        "            name=\"nested_radio_question\",\n",
        "            options=[\n",
        "                lb.Option(\"first_radio_answer\",\n",
        "                          options=[\n",
        "                              lb.Classification(\n",
        "                                  class_type=lb.Classification.Type.RADIO,\n",
        "                                  name=\"sub_radio_question\",\n",
        "                                  options=[lb.Option(\"first_sub_radio_answer\")])\n",
        "                          ])\n",
        "            ]),\n",
        "        lb.Classification(\n",
        "            class_type=lb.Classification.Type.CHECKLIST,\n",
        "            name=\"nested_checklist_question\",\n",
        "            options=[\n",
        "                lb.Option(\n",
        "                    \"first_checklist_answer\",\n",
        "                    options=[\n",
        "                        lb.Classification(\n",
        "                            class_type=lb.Classification.Type.CHECKLIST,\n",
        "                            name=\"sub_checklist_question\",\n",
        "                            options=[lb.Option(\"first_sub_checklist_answer\")])\n",
        "                    ])\n",
        "            ]),\n",
        "    ],\n",
        "    tools=[  # List of Tool objects\n",
        "        lb.Tool(tool=lb.Tool.Type.BBOX, name=\"bounding_box\"),\n",
        "        lb.Tool(tool=lb.Tool.Type.BBOX,\n",
        "                name=\"bbox_with_radio_subclass\",\n",
        "                classifications=[\n",
        "                    lb.Classification(\n",
        "                        class_type=lb.Classification.Type.RADIO,\n",
        "                        name=\"sub_radio_question\",\n",
        "                        options=[lb.Option(value=\"first_sub_radio_answer\")]),\n",
        "                ]),\n",
        "        lb.Tool(tool=lb.Tool.Type.POLYGON, name=\"polygon\"),\n",
        "        lb.Tool(tool=lb.Tool.Type.RASTER_SEGMENTATION, name=\"mask\"),\n",
        "        lb.Tool(tool=lb.Tool.Type.RASTER_SEGMENTATION,\n",
        "                name=\"mask_with_text_subclass\",\n",
        "                classifications=[\n",
        "                    lb.Classification(\n",
        "                        class_type=lb.Classification.Type.TEXT,\n",
        "                        name=\"sub_free_text\")\n",
        "                    ]\n",
        "                ),\n",
        "        lb.Tool(tool=lb.Tool.Type.POINT, name=\"point\"),\n",
        "        lb.Tool(tool=lb.Tool.Type.LINE, name=\"polyline\"),\n",
        "        lb.Tool(tool=lb.Tool.Type.RELATIONSHIP, name=\"relationship\")\n",
        "    ])\n",
        "\n",
        "ontology = client.create_ontology(\"Image Annotation Import Demo Ontology\",\n",
        "                                  ontology_builder.asdict(),\n",
        "                                  media_type=lb.MediaType.Image\n",
        "                                  )"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Step 3: Create a labeling project\n",
        "Connect the ontology to the labeling project"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Project defaults to batch mode with benchmark quality settings if this argument is not provided\n",
        "# Queue mode will be deprecated once dataset mode is deprecated\n",
        "project = client.create_project(name=\"Image Annotation Import Demo\",\n",
        "                                media_type=lb.MediaType.Image)\n",
        "\n",
        "project.setup_editor(ontology)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Step 4: Send a batch of data rows to the project\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "batch = project.create_batch(\n",
        "    \"image-demo-batch\",  # each batch in a project must have a unique name\n",
        "    global_keys=[\n",
        "        global_key\n",
        "    ],  # paginated collection of data row objects, list of data row ids or global keys\n",
        "    priority=1  # priority between 1(highest) - 5(lowest)\n",
        ")\n",
        "\n",
        "print(f\"Batch: {batch}\")"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Step 5: Create the annotations payload\n",
        "\n",
        "Create the annotations payload using the snippets of code above\n",
        "\n",
        "Labelbox support two formats for the annotations payload: NDJSON and Python Annotation types. Both are described below. If you are using Python Annotation types, compose your annotations into Labels attached to the data rows."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "### Python annotations\n",
        "\n",
        "Here we create the complete label ndjson payload of annotations only using python annotation format. There is one annotation for each reference to an annotation that we created."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "label = []\n",
        "annotations = [\n",
        "    radio_annotation,\n",
        "    nested_radio_annotation,\n",
        "    checklist_annotation,\n",
        "    nested_checklist_annotation,\n",
        "    text_annotation,\n",
        "    bbox_annotation,\n",
        "    bbox_with_radio_subclass_annotation,\n",
        "    polygon_annotation,\n",
        "    point_annotation,\n",
        "    polyline_annotation,\n",
        "    bbox_source,\n",
        "    bbox_target,\n",
        "    relationship\n",
        "] + cp_mask\n",
        "\n",
        "label.append(\n",
        "    lb_types.Label(data=lb_types.ImageData(global_key=global_key),\n",
        "                   annotations=annotations))"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "### NDJSON annotations\n",
        "Here we create the complete label ndjson payload of annotations only using NDJSON format. There is one annotation for each reference to an annotation that we created above."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "label_ndjson = []\n",
        "annotations = [\n",
        "    radio_annotation_ndjson,\n",
        "    nested_radio_annotation_ndjson,\n",
        "    nested_checklist_annotation_ndjson,\n",
        "    checklist_annotation_ndjson,\n",
        "    text_annotation_ndjson,\n",
        "    bbox_annotation_ndjson,\n",
        "    bbox_with_radio_subclass_ndjson,\n",
        "    polygon_annotation_ndjson,\n",
        "    point_annotation_ndjson,\n",
        "    polyline_annotation_ndjson,\n",
        "    bbox_source_ndjson,\n",
        "    bbox_target_ndjson,\n",
        "    relationship_ndjson,  ## Only supported for MAL imports\n",
        "] + cp_mask_ndjson\n",
        "\n",
        "for annotation in annotations:\n",
        "    annotation.update({\n",
        "        \"dataRow\": {\n",
        "            \"globalKey\": global_key\n",
        "        }\n",
        "    })\n",
        "    label_ndjson.append(annotation)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Step 6: Upload annotations to a project as pre-labels or ground truth\n",
        "For the purpose of this tutorial only import one of the annotations payloads at the time (NDJSON or Python annotation types)."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "Option A: Upload to a labeling project as pre-labels (MAL)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# upload MAL labels for this data row in project\n",
        "upload_job = lb.MALPredictionImport.create_from_objects(\n",
        "    client=client,\n",
        "    project_id=project.uid,\n",
        "    name=\"mal_job\" + str(uuid.uuid4()),\n",
        "    predictions=label\n",
        ")\n",
        "upload_job.wait_until_done()\n",
        "\n",
        "print(f\"Errors: {upload_job.errors}\")\n",
        "print(f\"Status of uploads: {upload_job.statuses}\")"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "Option B: Upload to a labeling project using ground truth"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Relationships are not supported with LabelImport\n",
        "# For this demo either run MAL or Ground Truth, not both\n",
        "\n",
        "# Upload label for this data row in project\n",
        "# upload_job = lb.LabelImport.create_from_objects(\n",
        "#     client = client,\n",
        "#     project_id = project.uid,\n",
        "#     name=\"label_import_job\"+str(uuid.uuid4()),\n",
        "#     labels=label)\n",
        "\n",
        "# print(\"Errors:\", upload_job.errors)\n",
        "# print(\"Status of uploads: \", upload_job.statuses)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# project.delete()"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    }
  ]
}